METHOD AND APPARATUS FOR A VIDEO 
PARTICLE EXPLOSION EFFECT 
FIELD OF THE INVENTION 

The invention relates to a real time 3D video effect. More specifically, it relates to a 
particle explosion effect. 

BACKGROUND OF THE INVENTION 

The hardware solutions currently proposed on the market for a particle explosion effect 
are based on dedicated chips that perform fixed functions at a fixed resolution. This 
3 approach requires that all effects be programmed in the chip which includes only a 
| limited set of attributes that can be applied by the user. Drawbacks to such an approach 

■? include: 

~? 

& 1) Coding and validating every transformation of the effect in the chip is a lengthy 

J process, so typically only a small set of parameters is provided. As a result, the editor 

3 has limited creative freedom. 

3 2) To keep the cost of DVE (Digital Video Effects) products as low as possible, 

i manufacturers often use simplified algorithms to minimize the complexity of their chips. 

= Quality is therefore compromised. 

3) Particle shapes are simple and hard coded. Users have no possibility of defining their 
own shapes. 

The traditional software solutions for particle explosion effects give the user many 
parameters to change the behavior of the particles. However, the shapes are quite 
simple (triangles, squares, etc.) and the user does not have the ability to modify them. 
Furthermore, the CPU computations required are too important to perform such effect in 
real time. 

The current approach to using 3D graphics technology for video effects is based on 
rendering. A combination of an effects algorithm, CPU power, and graphics 
acceleration, processes the video as quickly as possible, yet this approach does not 
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produce a real time Particle Explosion engine. The processing might be slower than real 
time and is therefore not reliable for streaming video at exactly 60 fields/sec (50 
fields/sec in PAL). 

[008] The main problems of all the previous approaches are: 

[009] 1 ) No possibility for the users to define their own particle shapes; 

[010] 2) Lack of flexibility in the effect parameters; 

[011] 3) Fully programmable effect does not run in real time. 

SUMMARY OF THE INVENTION 

[012] c ? Consequently, it is an object of the present Particle explosion effect invention to provide 
CQ a real time solution of a 3D Particle Explosion Video Effect based on a flexible particle 
\q system and user-defined shapes. 

TO 

[013][rf The present invention provides a real time and flexible solution of a 3D Particle 
b " Explosion video effect. 

[014]"i The flexibility is given to the user in one or both of two ways: 

It 1 - A powerful conversion tool allows the user to develop quickly any desired particle 
J 3 pattern. The Particle Explosion effect can extract the particle shapes from a simple 
image file that can be created by the user. The shapes are first drawn, using a paint 
application like PhotoShop. Color-coding is used to define the border of each particle, 
the order in which particles will start moving and the softness applied to the particle 
edges. The particle explosion effect creation tool is then used to convert the image file 
into the native Real time video editing system file needed, called a GFX file. The GFX 
file contains the alpha shape of each particle as well as the complete description of the 
mesh (list of triangles) required for rendering each particle. 

[015] 2 - The particle system algorithm drives the particles path with real-world keyframable 
attributes such as direction, dispersion and gravity. At each field, each new particle 
location in the 3D space is computed accordingly to these keyframes. The triangles are 
then processed with two rendering stages: one for the alpha shape of the particle and 



-2- 



1 0442-1 7us JA/IC 



* 



one for the video (or graphic) input. 

[016] A great flexibility is also given to the effect developer. The fact that this architecture is 
fully programmable means that additional parameters can be added with supplementary 
software, enabling the possibility to improve the Particle explosion effect at a very low 
cost. 

[017] According to a first broad aspect of the present invention, there is provided a method for 
generating a plurality of object definition data sets for a video particle explosion effect. 
The method comprises providing a graphics image data file of a particle pattern defining 
a shape of a plurality of particles; generating a plurality of object definition data sets 
£3 using the graphics image data file; wherein the object definition data sets can be used 
eB to render a particle explosion effect on a video file. 



[01 9] in Preferably, the graphics image data file further comprises defining at least one 
/ parameter of an explosion sequence for all of the plurality of particles, a spin parameter 
H for each of the plurality of particles and a softness of edges of each the plurality of 



C3 particles and wherein the shape, the explosion sequence, the spin parameter and the 
K softness are each defined in one of the plurality of channels. 

[020] According to another aspect of the present invention, there is provided a method for 
rendering a video particle explosion effect on a video source data file comprising: 
providing a graphics image data file of a particle pattern defining a shape of a plurality of 
particles; generating a sequence of object definition data sets using the graphics image 
data file; providing a video source data file; rendering the video particle explosion effect 
using the object definition data sets and the video source data file. 

[021] Preferably, the rendering comprises loading each field of the video source data file into 
a graphics engine; loading a corresponding one of the sequence of object definition 
data sets into the graphics engine; generating a particle exploded video output using the 
field and the corresponding object definition data sets. 



[018]£Q 



Preferably, the graphics image data file has a plurality of channels. 



BRIEF DESCRIPTION OF THE DRAWINGS 
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[022] These and other features, aspects and advantages of the present invention will become 
better understood with regard to the following description and accompanying drawings 
wherein: 

[023] FIG. 1 presents an overview of the system architecture; 

[024] FIG. 2a, 2b, 2c and 2d are schematic representations of an image file describing a 
pattern for a particle explosion effect comprising the red channel for shape description, 
the green channel for explosion order description, the blue channel for particle spin and 
the alpha channel for softness description, respectively. FIG. 2a, 2b, 2c and 2d contains 
photographs; 

f 3 

[025]; 3 FIG. 3a, 3b and 3c are schematic representations of the particle shape extraction and 

m triangle separation processes, wherein FIG. 3a is the extraction of one particle from the 

fy 

Co red component, 3b is the separation of the shape into small triangles and 3c is the 
§fj storage of the alpha shape of each small triangle in a data file. FIG. 3a, 3b and 3c 
contain photographs; 

[026] FIG. 4 shows a flow chart of the steps to convert an image file to a data file; 

[027] H FIG. 5 is a schematic representation of the particle explosion effect rendering operation. 

13 FIG. 5 contains photographs; and 

* . 

[028] FIG. 6 shows a flow chart of the steps to realize a particle explosion effect. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[029] While illustrated in the block diagrams as ensembles of discrete components 
communicating with each other via distinct data signal connections, it will be understood 
by those skilled in the art that the preferred embodiments are provided by a combination 
of hardware and software components, with some components being implemented by a 
given function or operation of a hardware or software system, and many of the data 
paths illustrated being implemented by data communication within a computer 
application or operating system. The structure illustrated is thus provided for efficiency 
of teaching the present preferred embodiment. 
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The real time capability of this effect is made possible by a Real time video editing 
system architecture and described in commonly assigned co-pending U.S. patent 
applications: US 09/653,701, US 09/653,703 and US 09/653,704, all incorporated 
herein by reference. 

The Real time video editing system architecture incorporates a real time-streaming 
approach that ensures that video fields are always processed at a fixed rate for all 
effects (60 fields per second in NTSC, 50 fields per second in PAL). Broadcast quality 
effects are achieved using special video enabled features that are built into a general- 
purpose 3D accelerator chip. This computer architecture is capable of doing real time 
A/B roll with 3D DVE and compositing. Two or more video streams enter a Input port 
(time multiplexed) and an output stream outputs from an output port. The 3D graphics 
engine does real time 3D processing on the multiple input streams to composite an 
output stream in real time. In the case of the preferred embodiment, the video effect is 
implemented within a Video Effect Filter. 

Figure 1 is a schematic block diagram of the real time video editing system and the Real 
time video editing system architecture. The real time video editing system uses high 
performance 3D graphics chips to perform a multitude of video special effects and 
compositing. Within the context of the system architecture of this invention, the 2D and 
3D command functions found in standard high performance graphics chips can be used 
for video processing. These commands can be considered as a set of basic 
instructions. By combining these commands with the correct parameters and in the 
correct sequence, virtually any visual effects can be achieved in real time. 

A real time video stream source 10, such as a camera or a video tape recorder, is 
connected to a video decoder 1 1 whose uncompressed digital stream output signal is 
fed into a CODEC 12 on a first, dedicated, video bus 40. The CODEC 12 is capable of 
video stream compression and decompression in real time. The CODEC 12 can also 
receive compressed video stream from an external device 37 such as Fire Wire (IEEE 
1394) (DV Camcorder) or SDTI. In another embodiment, the architecture comprises a 
codec capable of decompressing two video streams in real time. This external device 37 
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is connected to a compressed source interface 38 which is connected to the source 
controller 29. In the preferred embodiment the compressed source interface converts 
incoming signals from the compressed source interface 38 into DV25 or MPEG-2 
format. In another embodiment, the architecture comprises a compressed source 
interface 38 for each available compressed video source 37. The first video bus 40 
allows the decoder 11, the CODEC 12 and the video encoder 27 to communicate an 
uncompressed real time video stream. The CODEC 12 is connected to video bus 41 
which allows the CODEC to output in real time video fields of two uncompressed video 
streams to the input ports 13 and 14 of the video stream input buffers 15 and 16 of the 
graphics accelerator chip 36. In the preferred embodiment, a single bus TDMNX (with 
time division multiplexing to support multiple streams of video) is used. The input ports 
13 and 14, and the video stream input buffers 15 and 16 are controlled by a video input 
buffer controller 19. The goal of the input ports 13 and 14 is to select memory locations 
in each video stream input buffers 15 and 16 according to the video input buffer 
controller 19. These memory locations will receive the video streams. In the preferred 
embodiment, the graphics chip 36 is preferably a Matrox G400. 

The CODEC 12 is controlled by the source controller 29. In the preferred embodiment, 
the CODEC 12 is a C-Cube DV25/MPEG2. The source controller 29 has access to a 
video data storage 31. The source controller 29 also controls the video input buffer 
controller 19. An input buffer controller 18 controls the graphic input buffer 17 which is 
connected to the 3D rendering engine. The 3D rendering engine receives data streams 
from the video stream input buffers 15 and 16. 

Although it will be appreciated that various architectures for video editing software are 
possible, there is illustrated in Figure 1 a video editing controller 32 which represents 
the video editing user interface and controller processing associated with the application 
level. The video editing controller 32 may comprise, for example, the Adobe Premiere™ 
software package. For each specific video effect made available according to the 
preferred embodiment, a video effect filter module 20 is provided. The video effect 
control module 30 commands the source controller 29 to retrieve specific video data 
files from the video data storage 31. The video effect control module 30 also commands 
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the input buffer controller 18 and the output buffer controller 33. The output buffer 
controller 33 can generate an address signal which specifies a current field address. 
This current field address is sent to the output port 26, which will retrieve the 
corresponding field from the output buffer 25. The corresponding field can be then 
transmitted to the video encoder 27 and then to the display 28. 

The video effect control module 30 provides keyframes to the desired one of the video 
effect filters 20, 20', 20" etc.. In response, the video effect filter 20 sends commands to 
the graphics engine command interface 21 of the graphics chip 36 to implement the 
desired video processing within the graphics chip 36. The video effect control module 
30 also sends commands to the graphics engine command interface 21. The keyframe 
data describes the functionality of the effects and the graphics engine command 
interface 21 interprets the keyframe data to determine what specific graphics engine 
command is required. In the case of a video processing task requiring the capabilities of 
the 3D rendering engine 24, the graphics engine command interface 21 specifies the 
triangle and triangle meshes on which to map a video field with a resulting 3D 
perspective correct or non-perspective correct effect. Such an object is stored in 
memory 22 by data signal 42 and defined by the parameters for the vertices of the 
triangle or triangle meshes on which the video is to be mapped. Using the command 
data signal 45, the graphics engine command interface 21 is instructed to invoke either 
the 3D rendering engine 24 or the 2D graphics engine 23 to execute the desired video 
effect function and provide a result field for output buffer 25. 

In the case of the 3D rendering engine 24, the graphics engine command interface 21 
provides a specific command to the 3D engine 24 to map a selected one of the fields 
taken from one of the buffers 15, 16 and 17, thus obtaining a transformed field. 

In the case of the preferred embodiment, the video effect is implemented within the 
video FX filter 20. The particle effect is an effect by which a first image is exploded into 
particles to either transition to a second image or to solely filter the first image. Although 
the current invention preferably makes use of the Direct 3D technology developed by 
Microsoft, it will be understood by those skilled in the art that any other graphics 
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technology could be used. 

[039] The Particle Explosion effect of the present invention is able to extract the particle 
shapes from a simple graphic file that can be created by the user. This graphic file 
describes the shape of each particle. The file could also give information about other 
particularities of the particle explosion effect such as explosion sequence, rotation axis, 
particle spin and shape softness. 

[040] An image file that defines the particle shape, explosion sequence, spin and softness 
could be constructed in many different ways. What follows is a description of one 
preferred way of combining all of the preferred information for each particle of the 
particle explosion effect in one image file. The effect shown as an example is a 

□ 

"shattered glass" pattern. As will be readily understood, any pattern can be used to 
^ create an effect. In fact, any pattern comprising distinct components can be used with 
CO the particle explosion effect. Preferably, the image file is described as follows: 

W Bits depth: 32 bits 

"-J 

^ Size: 720x480 (NTSC) or 720x576 (PAL) 

[041] ^ The red channel defines the particle shape. Each particle must be drawn with a different 

O color. Since there are 8 bits per channel, up to 256 particles (2 8 ) can be defined. Colors 
[l 251 to 254 are reserved for special particles that are called "Background particles". 
Special behaviors (fade, inverse direction) will be available in the keyframe for these 
Background particles. Color 255 is reserved for invisible particle. FIG. 2a shows an 
example of the red channel for a shattering glass pattern. Each portion of the shattered 
glass is drawn in the red channel with a different color. 

[042] The green channel defines the explosion sequence. In this channel, each particle 
defined in the red channel will be associated with an explosion delay. Delay is 
represented by a value between 0 and 255. A zero delay will make the particle explode 
at the beginning of the effect progression, and a 255 delay will make it explode at the 
end. Intermediate values (between 0 and 255) can be used to define the desired 
explosion sequence. FIG. 2b shows an example of the green channel for a shattering 
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glass pattern. In this case, the closer the average value of the particle of the shattered 
glass pattern to the center of the image, the faster it will explode. 

[043] The blue channel defines the spin. In this channel, a number of rotations done by each 
particle is represented by a color between 0 and 255. A particle with color 0 will not spin 
at all and a particle with color 255 will have the maximum spin (for example: 100 
rotations). Intermediate values (between 0 and 255) can be used to define the desired 
spin. FIG. 2c shows an example of the blue channel for a shattering glass pattern. Each 
portion of the shattered glass pattern is given a different color to represent spin. 

[044] The alpha channel defines the alpha mask for each particle described in the red 
channel. This alpha mask is usually used to apply a softness band at the edges of each 

C3 

%2 particle. The alpha range is from 0 to 255. The zero value corresponds to fully invisible 
fjj and the 255 value corresponds to fully opaque. Usually, a 10 pixels softness width is 
*f enough to perform a good anti-aliasing. The softness is made with an alpha gradient 
m from invisible to opaque. This gradient will soften the video source edge with the 
underneath layer. FIG. 2d shows an example of the alpha channel for a shattering glass 
[ 3 pattern. Each particle is given an approximate 10 pixel softness band at its edges to 
Ca ensure that the edges will be soft at explosion. If no softness is given to the edge of the 
)i particles, staircase edges will appear on the edges of each particle and will affect the 
H quality of the effect. 

[045] It will be understood that parameters for the particles could be combined onto one 
channel. For example, a circle of a small diameter of a color different than the particle 
on which it is located could indicate an explosion sequence. The video effect controller 
would then have to recognize that circles of a particular diameter are not particles but 
rather define the explosion sequence of the particle on which they are located. Although 
this method less preferred, it would be useful in situations where less than 4 channels 
are available. 

[046] The conversion or preparation process step is done offline, within the particle explosion 
effect creation tool. This application can load the image file as described previously and 
process it with the Convert interface of the Particle explosion effect library. 
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[047] The Convert interface retrieves the relevant information from the image file and 
prepares two groups of data: 

[048] 1. A surface (2048x1024) called "Particle Matte" containing the alpha shape of all 
particles divided into small sections of 16x16 pixels and isolated from each other. Two 
triangles are defined in each 16x16 section. The smaller the triangles are the less 
unused pixels required for the processing of the image. Two copies of each section are 
written in the Particle Matte: a copy opaque of the shape extracted from the red 
channel and a copy with the softness from the alpha channel. FIG. 3a shows the first 
step of this conversion process. A particle is first extracted from the red component. An 
area encompassing all of its edges is delimited around the particle. FIG. 3b shows the 

□ 

h.i separation of the shape into small triangles. Finally, FIG. 3c shows the storage of the 

J ft alpha shape of each small triangle in the particle matte. 

in 

[049]*; 2. A particle description buffer containing the number of particles and useful information 
about each particle such as initial position, vertices, triangle faces, explosion delay 
extracted from the green channel, texture coordinates, etc. For each vertex (a triangle 
has 3 vertices) computed, three texture coordinate sets are stored : one set will be used 
to retrieve the particle opaque shape, another will be used to retrieve the particle shape 
with softness and the third one will be used to retrieve the video source texture. 



C3 



[050] These two groups of data are then stored by the application in a custom file format 
called GFX (Graphic effect). This file can be loaded by the Particle explosion effect 
Plugin within a video editing application (such as Adobe Premiere). The Plugin can then 
extract the Particle Matte (surface containing the particle shapes) as well as the particle 
description buffer and pass them to the Particle explosion effect Library at rendering 
time. 

[051] FIG. 4 is a flow chart which summarizes actions performed during the preparation step 
described previously. The 32 bit image file is first loaded 50. The three channels are 
scanned 52, for example, the red for the particle shapes, the green for the exploding 
sequence, the blue for the number of rotations and the alpha for the particle softness. 
The alpha shape is then stored in the Particle Matte 54. The triangles are computed and 
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stored. The vertices information is also stored in the particle description buffer 56. The 
Particle Matte and the ParticleSpecificBuffer are stored in a GFX file 58. 

[052] At rendering time, several ways can be thought of to control particles trajectory. For 
example, a particle system algorithm can be used to give real world properties to each 
particle. A particle system is a collection of particles, in which the dynamic behavior of 
the particles can be determined by the solution of sets of physical equations. 

[053] In such physical equations, we can take into account real world attributes such as 
position, speed and gravity. Further attributes can be used such as direction, dispersion, 
progression, spread, softness, etc. The position and the orientation of each vertex 
constituting the particles are computed at each field by the particle system algorithm 
depending on the effect progression and the keyframes provided by the user. As 
;H illustrated in FIG. 1, the user chooses, using the video editing controller 44, the 
SO parameters that are going to be used for the effect. The particles can be made to vanish 
in as they explode, the background particles can fade or go forward as the others are sent 
^ backwards, etc. 

E 

[054] H Before rendering the field result to the output surface, the video source, which has been 

□ selected by the user in the video effect control module 30, is transferred to a temporary 

□ surface with a blit command. The graphics engine 36 performs this blit command and all 
subsequent 2D and 3D operations. A border softness is then applied around the 
temporary surface in order to smooth the edges. Then, the particles can be rendered to 
the output surface. 

[055] Each particle effect field requires a rendering stage which is applied to all of the small 
triangles (16x16 pixels) of each particle effect field, as illustrated in FIG. 5. The particle 
explosion effect generator 70 receives the image files as inputs, processes the inputs 
and outputs the field of the effect applied on the video source file. 

[056] The particle explosion effect generator 70 receives the Alpha Argument 1 71, which is 
the GFX surface containing all of the alpha shapes of all the triangles constituting the 
particle shapes. The Alpha Argument 2 74, also inputted to the effect generator 70, is 
the transparency factor. The Alpha Argument 3 78 is, for example, the alpha channel of 
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the source image (or video) with border softness. Finally, the Color Input 79 is the 
image to which the particle explosion effect is to be applied. 

[057] Several choices of rendering states can be used to perform the effect. For example, 
anisotropic or tri-linear filtering can be used to improve the effect quality. Also, usage of 
a Z-Buffer could make sure that farther particles do not overwrite nearer particles. 

[058] The processing done by the particle explosion effect generator can be summarized with 
the following mathematical formulas which have been found to be the most efficient 
processing steps for the current invention: 

Alpha source = (Alphamput ^ * Alphai npu t2 * Alphai npu t3 ) 
□ Alpha out put- (Alpha S0U rce * SourceFactor ) + (Alpha des tination * DestinationFactor) 
Iq Color source = Colori n p U t 



Color out p U t= (Color source * SourceFactor ) + (CoIor destina tion * DestinationFactor) 
SourceFactor = Alpha2Source 
"J DestinationFactor = 1 - Alpha2Source 

[059]^ The source factor and the destination factor are there to ensure that a none-opaque 



C3 pixel will be blended with the destination pixel. Therefore, the background surface will 
tt be visible through the particle softness. 



[060] = As will be readily understood, other formulas could be used to render the effect without 
departing from the present invention. 

[061] The rendered effect for a particular time x is shown in 81. Using a wire frame overlay 
functionality 80, it is possible to see the triangles which were used to render the effect 
as shown in 82. 

[062] From this process is outputted a sequence of object definition data sets and the object 
definition data sets each correspond to a particle explosion effect on a video source file 
at a particular moment or time. The object definition data sets are associated with a 
particular field of the video source file by the graphics engine 36 to render the particle 
exploded video output. 
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[063] At the end of the rendering, a feedback command can be sent by the graphics engine 
36 to the video effect control module 30. The feedback command indicates that the 
engine 36 has completed the 2D and 3D operations. 

[064] FIG. 6 is a flow chart of the rendering operations described previously. The user first 
enters parameters 85. The selection of the video sources is then made 86. The video 
source is blit into a temporary surface 87. The border softness is applied to the video 
source 88. The particles are texture mapped with two texture stages, the particle 
shapes and the video source 89. Finally, the information that the graphics engine has 
completed the 2D and 3D operations is sent to the video effect control module 30. 

[065] The combination of the host CPU usage for the particle position computations and the 
□ 

h3 graphic chip usage for the 3D rendering make the Particle explosion effect able to 

f Q 

py perform in real time. Hence, no pre-rendering is required. 
[066]] ~ It should be noted that the present invention can be carried out as a method, can be 



embodied in a system, a computer readable medium or an electrical or electro- 
ns magnetical signal. 

n 

[067]=; It will be understood that numerous modifications thereto will appear to those skilled in 
3 the art. Accordingly, the above description and accompanying drawings should be taken 
as illustrative of the invention and not in a limiting sense. It will further be understood 
that it is intended to cover any variations, uses, or adaptations of the invention following, 
in general, the principles of the invention and including such departures from the 
present disclosure as come within known or customary practice within the art to which 
the invention pertains and as may be applied to the essential features hereinbefore set 
forth, and as follows in the scope of the appended claims. 
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